sycl: add Q2_K to DMMV reorder path - #25064
Merged
Merged
Conversation
github-actions
Bot
added
ggml
arthw
approved these changes
Jun 27, 2026
arthw
left a comment
arthw
left a comment
Contributor
There was a problem hiding this comment.
It's good job!
It could increase the perf on LLM Qwen3.5-27B-UD-Q2_K_XL.gguf.
Will you change it from draft to open status?
Or have you any plan to update this PR later?
Thank you!
Contributor
Author
|
Thanks, I did not realize you would review the draft PR. The performance improvement builds upon #25063, so I was waiting for that to merge before changing this one to open. I will mark it open now. I think there will be a merge conflict in either this or 25063 depending on which is merged first. |
malsbat
marked this pull request as ready for review
June 29, 2026 16:33
Contributor
|
OK, I will enable to merge #25063 as soon. |
Contributor
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
malsbat
force-pushed
the
q2-k-dmmv-reorder
branch
from
July 7, 2026 18:51
2dd185f to
ba90945
Compare
Contributor
Author
|
Rebased against current master branch |
arthw
added
the
merge ready
CowboyTim
pushed a commit
to aardbeiplantje/llama.cpp
that referenced
this pull request
Jul 21, 2026
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
smalinin
pushed a commit
to smalinin/llama.cpp
that referenced
this pull request
Aug 4, 2026
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
satindergrewal
pushed a commit
to satindergrewal/llama.cpp
that referenced
this pull request
Aug 12, 2026
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
mndodd
added a commit
to mndodd/llama.cpp
that referenced
this pull request
Aug 12, 2026
100 commits, 8 SYCL. Conflicts in 6 files, 10 hunks. 7 union-merged (independent globals/switch-cases: our reorder_in_gemm + NVFP4 alongside their fa_onednn + Q2_K ggml-org#25064). 3 needed a call: - fattn.cpp: keep our env-gated MMA override FIRST, then upstream's oneDNN SDPA prefill FA (ggml-org#25222), then our decode VEC/TILE doors. Our decode + quantized-KV path stays ours; oneDNN takes prefill-shaped only. WARN: ggml-org#25222 has the RIG-HYGIENE ggml-org#26 pool_alloc use-after-free -- needs multi-turn 2-GPU garbage-token validation before trust. - fattn-vec.hpp: take upstream ggml-org#25205 Battlemage nthreads=256, preserve our is_vec_kernel=true arg (selects nsm geometry in launch_fattn). - dequantize.hpp q2_K: keep our f32-arithmetic dequant (NMSE 2.5e-7). test-backend-ops.cpp: v=4 was overloaded -- upstream ggml-org#25064 uses it for non-cont-a-last-2-dim, ours for transposed-b. Moved our transposed-b to bit 1<<4 so both coverages survive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Add Q2_K to DMMV reorder path.
Additional information
The measured results are on top of #25063.
Requirements